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Abstract 

Pilot  judgment  errors  have  long  been  recognized  as  an  important  factor  in  aviation 
accidents.  Previous  studies  have  demonstrated  that  specialized  training  procedures 
can  significantly  reduce  the  number  of  decisional  errors  made  by  newly  certified 
private  pilots  during  in-flight  tests.  However,  the  subjects  in  these  studies 
were  all  college-age  students  enrolled  in  full  time  aviation  training  programs 
which  were  taught  by  highly  motivated  instructors.  The  present  study  examined 
the  utility  of  revised  judgment  training  materials  with  typical  private  pilot 
applicants  in  conventional  flight  school  settings  at  ten  fixed  based  operations 
within  the  FAA's  Eastern  Region.  The  performance  of  a  sample  of  subjects  who 
received  judgment  training  was  compared  with  that  of  a  control  group  drawn  frem 
these  same  FBOs.  The  behavioral  test  of  judgment  was  in  the  form  of  an  observation 
flight  administered  by  observers  who  were  uninformed  of  the  details  of  the 
experimental  design.  Students  and  instructors  also  completed  a  critique  of 
the  program  materials.  The  results  of  the  study  suggest  that  improvements  in 
pilot  decisionmaking  skills  can  be  achieved  in  the  less  formal  instructional 
climate  which  characterizes  many  conventional  flight  school  programs.  The  revised 
judgnent  training  program  and  instructional  materials  are  acceptable  to  the 
user  cocnnunity,  and  most  participants  found  them  to  be  very  useful. 
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PRIVATE  PILOT  JUDGMENT  TRAINING 
IN  FLIGHT  SCHOOL  SETTINGS 


INTRODUCTION 

In  1976,  the  Federal  Aviation  Administration  (FAA)  contracted  with 
University  of  Illinois  researchers  to  investigate  the  degree  to  which 
faulty  pilot  judgment  contributed  to  civil  aviation  accidents  and  to 
determine  whether  and  how  pilot  judgment  might  be  taught  and  evaluated. 
This  study  reviewed  all  general  aviation  accident  data  between  1970  and 
1974,  and  concluded  that  faulty  pilot  decisional  activities  were  involved 
in  35%  of  all  nonfatal  general  aviation  accidents  and  in  52%  of  all  fatal 
accidents  (Jensen  and  Benel,  1977).  The  authors  offered  the  following 
two-component  definition  of  judgment: 

1.  The  ability  to  search  for  and  establish  the  relevance  of  all 
available  information  regarding  a  situation,  to  specify 
alternative  courses  of  action,  and  to  determine  expected 
outcomes  from  each  alternative. 

2.  The  motivation  to  choose  and  authoritatively  execute  a 
suitable  course  of  action  within  the  time  frame  permitted  by 
the  situation  (1977,  p.  34) 

These  researchers  concluded  that  Dilot  judgment  could  be  taught  and 
objectively  evaluated,  and  they  proposed  some  ways  in  which  this  could  be 
accomplished.  Two  years  later,  the  FAA  contracted  with  Embry-Riddle 
Aeronautical  University,  Daytona  Beach,  Florida  (ERAU)  to  develop 
judgment  training  materials  for  student  and  instructor  pilots,  devise 
procedures  for  the  objective  measurement  of  pilot  judgment,  and 
demonstrate  the  effectiveness  of  such  training.  A  revised  definition  of 
oilot  judgment  was  developed  by  this  group 

Pilot  judqment  is  the  mental  process  bv  which  the  pilot 
recognizes,  analyzes,  and  evaluates  information  regarding 
himself,  the  aircraft,  and  the  outside  environment  The  final 
step  in  the  process  is  to  make  a  decision  pertaining  to  the  safe 
operation  of  the  aircraft  and  to  imDlement  the  decision  in  a 
timely  manner  (1982,  p  4) 

The  ERAU  project  (Berlin,  Gruber,  Holmes,  Jensen,  i_au,  Mills  ar.f 
O'Kane,  1982)  produced  prototype  student  and  instructor  pilot  training 
manuals,  and  developed  an  observation  flight  protocol  to  measure  pilot 
judgment  Using  this  technique,  the  ERAU  team  then  carried  cut  a  limited 
validation  study  of  the  training  materials  The  subjects  were  all  students 


Judgment  Training 


Page  2 


enrolled  in  an  ERAU  flight  program  and  had  a  mean  age  of  19  years.  One 
group  of  subjects  was  trained  under  the  conventional  ERAU  primary  flight 
program,  while  a  second  group's  flight  and  ground  training  incorporated  the 
newly  developed  judgment  materials.  Both  groups  were  evaluated  on  the 
observation  flight,  and  the  group  which  had  received  judgment  training  did 
significantly  better  than  the  control  group  on  this  behavioral  test. 

In  1982,  the  FAA  entered  into  a  joint  research  agreement  with 
Transport  Canada  to  further  examine  the  ERAU  prototype  materials  and  to 
independently  evaluate  judgment  training  in  the  Canadian  pilot  population. 
Several  experiments  have  thus  far  been  conducted  under  the  provisions  of 
that  agreement. 

The  subjects  in  the  initial  Canadian  study  were  civilian  Air  Cadets 
participating  in  a  summer  flight  training  program  in  the  Ontario  Region. 
All  were  in  their  late  teens  and  early  twenties,  and  more  than  half  of  the 
subjects  already  held  glider  licenses.  This  study  used  geographically 
isolated  experimental  and  control  groups,  and  the  observation  flight  was 
administered  under  circumstances  which  permitted  its  true  purpose  to  be 
better  disguised  than  in  the  ERAU  study.  Here  too,  those  subjects  who 
received  the  judgment  training  did  significantly  better  on  the  observation 
flight  than  did  control  subjects  (Buch,  1982;  Buch  and  Diehl,  1984).  The 
results  of  both  the  ERAU  and  Air  Cadet  studies  are  shown  in  Figure  1. 
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A  second  Transport  Canada  study  was  undertaken  in  order  to 
determine  whether  the  use  of  the  self-paced  student  manual  could  improve 
pilot  decision-making  skills  without  the  involvement  of  flight  instructors 
(Buch  and  Diehl,  1983).  This  study,  conducted  at  Confederation  College  of 
Applied  Arts  and  Technology  (Thunder  Bay,  Ontario)  used  a  group  of  17 
university  students.  All  were  licensed  private  pilots  with  "relatively 
little  flight  time",  Subjects  were  first  evaluated  using  a  well-disguised 
observation  flight  procedure,  then  given  the  self-paced  manuals  and 
instructed  to  complete  them  on  their  own.  Following  the  self-study 
program,  a  second  observation  flight  was  scheduled.  The  use  of  the 
judgment  training  manuals  alone  resulted  in  a  significant  improvement  in 
observation  flight  performance,  from  53%  on  the  pretest  to  62%  on  the 
post-test. 

Another  Canadian  study,  this  one  on  the  effectiveness  and  permanency 
of  judgment  training  over  time,  is  currently  under  way  at  a  community 
college  in  Chicoutimi,  Quebec  (Buch  and  de  Bagheera,  1985).  In  this  study, 
subjects'  aeronautical  decision-making  skills  will  be  evaluated  up  to  three 
years  after  their  initial  judgment  training. 

While  pilot  judgment  and  decision-making  may  be  related  to 
relatively  inflexible  aspects  of  pilot  personality  or  to  the  interaction  of 
personal  and  situational  variables  (Lester  and  Bombaci,  1984),  these 
experiments  strongly  suggest  that  the  use  of  the  judgment  training 
curriculum  can  significantly  improve  pilot  performance  and  enhance 
aviation  safety.  However,  there  were  a  number  of  limitations  to  the 
studies  which  compromise  the  external  validity  of  these  findings.  Most  of 
the  subjects  were  young  males  who  had  only  recently  completed  their 
secondary  education.  Both  the  ERAU  students  and  the  Canadian  Air  Cadets 
were  enrolled  in  highly  concentrated  aviation  training  programs  Most  of 
the  Air  Cadets  already  held  glider  licenses,  and  all  of  the  subjects  in  the 
Confederation  College  study  were  licensed  private  pilots  at  the  time  the 
judgment  training  was  begun 

it  was,  therefore,  felt  necessary  to  demonstrate  that  the  judgment 
training  curriculum  would  be  effective  with  the  general  population  of  ab 
initio  student  pilots  and  in  the  type  of  training  environments  typically 
encountered  at  operational  flight  schools  Further  work  was  also  needed 
in  order  to  produce  training  materials  which  were  acceptable  to  students 
and  instructors  in  these  settings 
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Thus,  following  completion  of  the  Air  Cadet  study,  it  was  decided  to 
undertake  a  major  revision  of  the  judgment  training  manuals,  and  then  to 
demonstrate  the  effectiveness  of  these  revised  materials  in  conventional 
flight  schools.  The  revision  of  the  student  pilot  and  instructor  manuals 
was  undertaken  as  a  joint  project  by  FAA,  Transport  Canada,  and  the 
General  Aviation  Manufacturers  Association  (GAMA)  (See  Note  I) 
Audio-visual  materials,  consisting  of  approximately  one  hundred  35  mm 
color  slides  and  a  six  minute  video  tape,  were  also  produced  to  enhance  the 
training  of  both  student  pilots  and  instructors. 

The  70  page  Student  Manual  introduces  the  concept  of  judgment  and 
describes  the  role  of  risk  assessment  and  decision-making  in  aviation. 
The  student's  attention  is  then  directed  to  the  subject  areas  of  Pilot, 
Aircraft  and  Environment,  and  to  Six  Action  Ways  or  modes  of  pilot  error 
(Do,  No  Do,  Early  Do,  Late  Do,  Under  Do,  Over  Do).  The  notion  that  errors  are 
cumulative  is  discussed  in  the  context  of  the  Poor  Judgment  Chain,  and  the 
student  is  given  a  method  to  help  break  this  chain.  Following  a  discussion 
of  the  Three  Mental  Processes  of  Safe  Flight  (Automatic  Reaction,  Problem 
Resolving,  Repeated  Reviewing),  a  Self-Assessment  Inventory  introduces 
the  student  to  Five  Hazardous  Thoughts  (Impulsivity,  Invulnerability, 
Macho,  Anti-Authority,  Resignation),  and  specific  Antidotes  to  each 
hazardous  thought  are  provided.  The  manual  concludes  with  chapters  on 
Stress  Reduction  and  some  specific  applications  of  pilot  decision-making 
skills 

The  63  page  Instructor  Manual  provides  specific  guidance  on  how  to 
teach  judgment,  and  outlines  18  in-flight  Lesson  Plans  and  50  Judgment 
Training  Scenarios  to  help  accomplish  the  program's  goals  (See  Note  2).  It 
includes  a  proposed  schedule  of  student  work,  suggestions  on  how  to 
manage  judgment  training,  and  a  master  set  of  student  records  to  be 
duplicated  and  used  at  each  training  site  The  manual  concludes  with  a  set 
of  Postcheck  Exercises  to  assure  that  the  student  has  mastered  the 
judgment  training  material  Both  of  these  manuals  are  extensively  revised 
versions  of  the  material  which  had  been  developed  by  the  ERAU  research 
team 
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METHOD 

The  experimental  design  compared  the  performance  of  two  groups  of 
newly  rated  private  pilots  on  a  behavioral  test  of  judgment.  The  study 
was  conducted  at  ten  FBOs  within  the  FAA's  Eastern  Region,  which 
consists  of  the  geographic  area  within  the  states  of  Delaware,  Maryland, 
New  York,  Pennsylvania,  West  Virginia,  and  Virginia,  and  the  District  of 
Columbia.  The  participating  FBOs  are  listed  in  Appendix  A.  The  control 
group  included  subjects  trained  at  these  FBOs  using  conventional  pilot 
training  programs.  The  experimental  group  used  subjects  drawn  from  these 
same  FBOs.  Their  training  followed  the  same  curriculum  as  was  used  in 
the  control  group,  but  with  the  addition  of  a  judgment  component  taught  by 
specially  trained  instructors.  The  behavioral  test  was  in  the  form  of  an 
obsci  vation  flight  which  was  administered  by  specially  trained  observers 
who  were  uninformed  of  the  details  of  the  experimental  design. 

Cooperating  instructors  at  each  of  the  ten  participating  FBOs  were 
individually  trained.  At  some  FBOs  only  one  or  two  instructors  were 
included,  while  at  others  the  entire  cadre  of  CFIs  agreed  to  participate.  A 
one  hour  slide  presentation  described  the  rationale  and  philosophy  behind 
judgment  training,  reviewed  the  content  and  methods  used  in  the 
curriculum,  and  presented  a  number  of  vignettes  which  illustrated  ways  in 
which  the  judgment  materials  could  be  integrated  into  flight  instruction. 
Instructors  were  then  given  the  Student  and  Instructor  Manuals  to  study, 
and  were  later  interviewed  in  order  to  assure  their  familiarity  with  the 
curriculum.  All  instructors  participating  in  the  program  were  unpaid 
volunteers  and  were  unaware  that  their  students'  judgment  might  later  be 
evaluated  An  FAA  Scientist  occasionally  visited  the  FBOs  to  coach 
instructors  and  monitor  the  progress  of  the  program. 

The  observation  flights  were  conducted  by  observers  who  were 
unaware  of  the  overall  experimental  design  and  who  had  no  knowledge  of 
which  type  of  training  each  subject  pilot  had  received.  The  observers 
were  employees  of  the  Aircraft  Owners  and  Pilots  Association  (AOPA)  and 
had  volunteered  for  the  project  Each  held  an  Airline  Transport  Pilot 
certificate,  along  with  Certified  Flight  Instructor,  instrument  and 
multi-engine  ratings.,  and  each  had  between  5,000  and  10,000  hours  of 
total  flight  time  During  training  sessions,  observers  were  introduced  to 
their  roles  in  the  study  it  was  stressed  that  their  primary  concern  was  to 
maintain  flight  safety  at  all  times,  while  keeping  an  unobtrusive  record  of 
the  subject's  performance  Training  underscored  that  the  observer's  role 
was  to  present  tasks  and  record  performance,  rather  than  to  instruct  or 
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critique  the  students.  Each  observer  was  assigned  two  or  three  airports 
from  which  their  observation  flights  were  to  be  conducted. 

During  individual  familiarization  flights  at  each  airport,  specific 
terrain  features  and  targets  to  be  used  as  checkpoints  during  the 
observation  flights  were  selected.  This  flight  was  also  used  as  an 
opportunity  for  further  training  of  the  observer  in  the  presentation  and 
scoring  of  the  observation  flight  scenarios.  Because  the  terrain,  weather, 
traffic  conditions,  and  targets  varied  considerably  among  the  ten  airports, 
no  attempt  was  made  to  establish  a  rigid  flight  profile  which  would  apply 
to  all  observation  flights.  But  each  observer  flew  with  subjects  from  both 
the  experimental  and  control  groups  and  thus  acted  as  his  own  blind 
control  for  these  factors. 

The  behavioral  test  of  pilot  judgment  was  in  the  form  of  a  12  item 
observation  flight  that  could  be  safely  and  reliably  presented  at  all 
evaluation  sites  Items  were  selected  to  cover  pilot  activities  normally 
occurring  during  the  preflight,  departure,  cruise  and  arrival  phases  of  a 
typical  general  aviation  flight  conducted  under  visual  flight  rules.  Each 
item  required  a  decision  by  the  pilot,  and  provided  a  measure  of  judgment 
rather  than  of  knowledge,  skill  or  experience  The  12  items  and  the 
criterion  behavior  which  constituted  "good  judgment"  for  each  is  presented 
in  brief  form  in  Table  1.  A  complete  description  of  the  specific 
procedures. for  administering  each  item  and  the  scoring  guidelines  used  by 
the  observers  appears  in  Appendix  B. 


Table  1.  Observation  flight  items. 


1.  Pilot  checks  weather  and  Notices  to  Airmen  prior  to  boarding 
aircraft. 

2  Durinq  prefliqht  inspection,  pilot  detects  and  remedies  loose 
fuel  tank  cap,  loose  oil  filler  cap,  or  popped  circuit  breaker 

3.  Prior  to  engine  start,  pilot  detects  and  removes  an  empty 
soft  drink  bottle  or  flashlight  from  cockpit  floor,  or  removes  a 
hat  placed  on  glareshield. 

4  Pilot  shuts  down  engine  or  directs  observer  to  remain  in 
aircraft  when  observer  indicates  his  intention  to  exit  aircraft 
in  order  to  check  something  (eg.  loose  cowling)  shortly  after 
enqine  start 
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5.  Pilot  requests  that  observer  hold  questions  and  comments 
following  interruptions  during  preflight,  taxi,  or  run-up 
checklist. 

6.  Pilot  declines  observer  request  to  make  a  marginally  safe 
intersection  takeoff,  non-standard  turn  from  pattern,  or  low 
altitude  turn  immediately  after  takeoff. 

7.  Pilot  declines  observer  request  to  overfly  a  target  structure 
(eg.  friend’s  house)  in  a  built-up  area  by  barely  1,000  feet. 

8.  Pilot  declines  observer  request  to  execute  a  low  steep  turn 
over  a  target  structure. 

9.  Pilot  declines  observer  request  to  fly  at  an  unsuitably  low 
altitude  over  water,  mountainous,  or  heavily  wooded  terrain 

10  Pilot  adjusts  altitude  to  comply  with  VFR  cruising 
standards  following  observer  request  to  turn  to  new  heading 

1 1.  Pilot  avoids  controlled  airspace  or  obtains  ATC  clearance 
following  observer  request  for  flight  into  active  Airport 
Traffic  Area. 

12.  Pilot  declines  observer  request  to  fly  a  non-standard 
pattern,  directly  over  airport  at  pattern  altitude,  hiqh  final 
approach  leg,  or  downwind  landing  at  uncontrolled  airport 

!ii  Gi  {let  to  per  in  it  a  tealislic  assessment  oi  pilot  judgment,  it  was 
necessary  to  disguise  the  true  purpose  of  the  study  Thus,  the  observation 
flight  was  described  to  subjects  as  being  part  of  an  "aviation  safety 
survey"  being  conducted  by  AOPA.  Subjects  initially  received  a  letter 
stating  that  volunteers  were  being  asked  to  participate  in  a  short 
cross-country  flight  in  connection  with  the  planned  revision  of  sectional 
aeronautical  charts  The  incentive  for  these  newly  licensed  pilots  was 
they  would  be  given  the  opportunity  to  fly  a  rented  aircraft  from  their 
home  airport  free,  in  exchange  for  their  participation  in  this  study  The 
letter  requested  that  pilots  call  a  toll-free  telephone  number  if  they 
were  interested  in  participating  When  a  subiect  telephoned,  they 
generally  spoke  with  the  observer  who  would  conduct  their  observation 
flight  The  observer  introduced  himself,  explained  the  proposal  in  greater 
detail  and,  if  the  subject  was  still  willing,  made  arrangements  to  meet  at 
the  pilot's  home  airport 

Upon  arrival  at  the  airport,  the  subject  was  greeted  by  a  casually 
dressed  observer  who  explained  the  study  of  the  revised  sectional  charts 
and  the  need  for  evaluations  from  newly  rated  private  pilots  The  subiect 
was  told  that  he  would  first  complete  a  questionnaire  on  a  prototype 
chart,  and  then  fly  the  survey  flight  using  the  standard  chart  tor  his  hume 


Judgment  Training 


Page  8 


area.  It  was  explained  that  half  of  the  observation  flight  would  be  flown 
at  low  altitudes,  with  the  other  half  at  cruise  altitudes.  Subjects  were 
told  that  the  major  concern  during  the  flight  was  how  useful  the  currently 
used  chart  symbols  were  for  a  new  VFR  pilot.  The  observer  explained  that 
the  flight  would  proceed  with  the  subject  being  asked  to  locate  certain 
targets.  The  route  of  the  flight  was  then  shown,  and  the  subject  was 
reminded  that  he  was  the  pilot-in-command  and  was  to  make  all  decisions 
regarding  the  flight.  The  subject  was  then  given  the  prototype  Los  Angeles 
sectional  chart  along  with  a  22  item  "National  Airspace  Review  VFR 
Prototype  Chart  Evaluation  Form"  (See  Note  3).  Subjects  were  allowed  as 
much  time  as  needed  to  study  the  prototype  chart  and  complete  the 
questionnaire.  They  then  could  proceed  to  flight  planning  activities  and 
the  preflight  aircraft  inspection. 

Observers  did  not  volunteer  detailed  information  about  their  own 
flying  background.  If  a  subject  inquired,  the  observer  told  them  only  that 
he  was  a  rated  pilot.  If  pressed  for  more  details,  the  observer  allowed 
that  he  "used  to  be  a  flight  instructor"  but  was  not  actively  involved  as 
such  any  longer  This  satisfied  virtually  all  subjects  and  helped  to 
minimize  the  tendency  to  attribute  authority  fiqure  status  to  the 
observers.  Items  were  presented  by  the  observers  in  such  a  manner  as  to 
never  require  them  to  take  control  of  the  aircraft  or  to  divulqe  the  true 
purpose  of  the  fliqht.  Each  item  was  scored  and  unobtrusively  recorded  by 
the  observer  at  the  time  it  was  presented  according  to  the  specific 
scoring  guidelines  using  a  dichotomous  "good  judgment"  or  "poor  judgment" 
scoring  system. 

Following  completion  of  the  private  pilot  check  ride,  each  subject 
who  had  received  judgment  training  was  asked  to  complete  a  12  item 
evaluation  of  the  program  (See  Appendix  C)  Instructors  completed  a 
similar  16  item  form  (See  Appendix  D). 
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RESULTS  &  DISCUSSION 


Subjects 

The  subiects  in  the  control  group  were  tested  by  one  of  four 
observers  When  the  control  group  data  were  examined,  significant 
differences  between  observers  in  the  mean  observation  flight  scores  of 
their  respective  subjects  were  noted.  Given  these  differences,  an  effort 
was  made  to  have  each  observer  fly  with  about  the  same  number  of 
subjects  from  both  the  experimental  and  control  groups.  However, 
because  of  scheduling  difficulties,  only  three  of  the  four  observers  were 
able  to  fly  with  experimental  group  subjects.  For  this  reason,  the 
observation  flight  performance  of  20  subjects  from  the  experimental 
group  is  compared  with  that  of  the  25  control  group  subjects  who  were 
tested  by  these  same  three  observers. 

It  was  not  possible  to  schedule  an  observation  flight  with  any 
subjects  at  one  of  the  ten  FBO's.  Also  excluded  from  the  analysis  is  data 
from  one  subject  who  appeared  for  the  observation  flight  in  a  personally 
owned  complex  turbo-charged  aircraft.  The  number  of  subjects 
participating  in  experimental  and  control  group  observation  flights  from 
each  of  the  ten  FBO's  is  shown  in  Appendix  E.  The  data  for  the  control 
group  appears  in  Appendix  F,  while  experimental  group  data  is  contained  in 
Appendix  G. 

No  significant  difference  between  experimental  and  control  group 
subjects  was  noted  with  respect  to  age  (t=0. 1 5,  df=42).  For  the  24  control 
group  subjects  on  whom  demographic  information  was  available,  the  mean 
age  was  34.0  years,  with  a  range  from  20  to  64  years.  The  20 
experimental  group  subjects  ranged  in  age  from  18  to  55  years,  with  a 
mean  age  of  34.4  years. 

There  were  also  no  significant  differences  between  the  experimental 
and  control  groups  in  the  number  of  flight  hours  or  the  number  of  months 
spent  in  training  for  the  private  pilot  certificate  Subjects  in  the 
experimental  group  (N=20)  had  a  mean  of  90  6  hours  of  flight  time  when 
they  passed  the  private  pilot  check  ride,  while  control  group  subjects 
(N=  1 8)  had  a  mean  of  72  4  hours  (t=1  42,  df =36).  For  the  control  group 
subjects  (N=  1 7),  an  average  of  9  6  months  elapsed  between  their  first  solo 
and  the  check  ride,  compared  with  a  mean  of  7  7  months  in  the 
experimental  group  (N=20)  (t=0  73,  df =35) 

No  significant  differences  between  the  experimental  and  control 


Judgment  Training 


Page  10 


groups  were  noted  in  the  amount  of  time  which  elapsed  between  the 
private  pilot  check  ride  and  the  observation  flight  (t=0.54.  df =35).  On 
average,  control  group  subjects  (N=  1 7)  were  tested  5,4  months  following 
the  check  ride.  Experimental  group  subjects  (N=20)  were  tested  after  6.6 
months, 

There  was  no  significant  difference  between  the  experimental  and 
control  groups  on  the  scores  earned  when  they  passed  the  written 
examination  for  the  private  pilot  certificate  (t=0.36,  df=42).  Subjects  in 
the  control  group  (N=24)  earned  a  mean  score  of  87.5,  while  experimental 
group  subjects  (N=20)  had  a  mean  score  of  88.5  on  this  examination.  Two 
subjects,  one  in  the  control  group  and  one  in  the  experimental  group,  did 
not  pass  the  written  examination  on  the  first  attempt  One  subject  in  the 
control  group  did  not  pass  the  private  pilot  practical  test  on  the  first 
attempt 


Thus,  any  difference  between  the  experimental  and  control  groups  in 
observation  flight  performance  cannot  reasonably  be  attributed  to 
differences  in  aqe,  flight  experience,  aeronautical  knowledge,  or  skill. 

Observation  Flight  Performance 


The  experimental  group,  which  received  judgment  training,  did  about 
10%  better  on  the  observation  flight  than  did  the  control  group.  This 
difference  is  statistically  significant  (t=2. 13,  df=43,  p<.05),  and  suggests 
that  the  judgment  training  program  is  effective  in  conventional  flight 
school  settings.  However,  the  improvement  in  pilot  judgment  may  be 
considerably  less  in  these  settings  than  was  suggested  by  previous 
studies.  Results  of  the  observation  flight  are  presented  in  Table  2. 


Table  2.  Observation  flight  results. 
Correct  Responses 


Control 

Group 

Experimental 

Group 

(PERCENT) 

599 

Mean 

70.0 

163 

5.  D 

15.1 

25 

N 

20 
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There  was  considerable  variability  in  the  mean  observation  flight 
scores  assigned  to  subjects  by  each  of  the  three  observers,  as  well  as 
great  differences  among  subjects  from  different  FBO's.  Because  of  the 
small  cell  frequencies  and  the  partial  confounding  of  observers  with  FBO's, 
it  was  not  possible  to  statistically  isolate  the  effects  of  these  extraneous 
variables.  However,  these  influences  were  examined  in  two  ways.  When 
the  observation  flight  performance  of  the  experimental  and  control  group 
subjects  run  by  each  of  the  three  observers  are  compared,  improvements 
of  between  four  and  eight  percent  are  noted  in  the  case  of  each  observer. 
These  results  are  shown  in  Table  3. 


Table  3.  Mean  observation  flight  score  by  observer. 


Observer  A 

Observer  B 

Observer  C 

(N= 1 1 ) 

(N=20) 

(N=  1 4) 

Control  Group 

47.6 

61.5 

73.5 

Experimental  Group 

52.7 

65.8 

81.3 

Moreover,  when  the  overall  analysis  is  restricted  to  those  FBO's 
which  were  represented  in  both  the  experimental  and  control  groups,  the 
improvement  in  observation  flight  performance  which  results  from 
judgment  training  is  noted  to  be  even  greater  (t-2.22,  df =28,  p< .05)  This 
comparison  is  shown  in  Table  4. 

Table  4.  Observation  flight  results  for  FBO's 
represented  in  both  experimental  and  control  groups 

Correct  Responses 


Control 

Group 

Experimental 

Group 

(PERCENT) 

65.7 

Mean 

77  8 

15.6 

5.  D 

’.3  6 

17 

N 

!  3 

These  supplemental  analyses  suggest  that  the  finding  that  judgment 
training  improves  observation  flight  performance  is  quite  reMabie  m  spite 
of  the  high  variability  among  ^BO's  and  between  observers 
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Judgment  Training  Materials 

Twenty-five  pilots  who  received  judgment  training  provided  critiques 
of  the  training  program  and  materials  Forty-two  participatinq 
instructors  also  evaluated  the  program  Because  the  Student  Manual 
represented  a  major  revision  of  the  ones  used  in  both  the  ERAU  and  Air 
Cadet  studies,  it  was  particularly  important  to  establish  its  user 
acceptability  in  this  manner.  These  student  and  instructor  critiques  were 
quite  helpful  in  identifying  the  strengths  and  weaknesses  of  the  various 
program  components.  As  may  be  seen  in  Figure  2,  the  Student  Manual  was 
very  well  received.  More  than  half  of  both  the  student  and  instructor 
samples  indicated  that  the  Student  Manual  was  "very  useful",  while  12%  of 
the  students,  and  none  of  the  instructors,  felt  that  it  was  "not  at  all 
useful". 

instructor  evaluations  of  their  teaching  materials  was  also  favorable, 
but  not  quite  as  positive  as  in  the  case  of  the  Student  Manual  As  may  be 
seen  in  Fiqure  3,  43%  felt  that  the  Instructor  Manual  as  a  whole  was  "very 
useful",  while  32%  gave  high  ratings  to  the  m-flight  Lesson  Plans  and  the 
Judqment  Training  Scenarios  However,  10%  of  the  respondents  felt  that 
the  Instructor  Manual  was  only  "slightly  useful",  and  14%  saw  little  value 
in  the  Lesson  Plans  and  Judqment  Training  Scenarios 


VERY  USEFUL 


nODERATELY  USEFUL 


SLIGHTLY  USEFUL 


NOT  AT  ALL  USEFUL 
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Figure  3.  Ratings  of  instructional  materials 


Questions  in  both  the  student  and  instructor  critiques  were  directed 
at  five  basic  components  of  the  judgment  training  program  Figures  4 
through  8  present  the  proportions  of  each  respondent  group  which  gave 
ratings  of  "useful",  "moderately  useful",  "slightly  useful",  and  "not  at  all 
useful"  to  each  of  these  program  elements 
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Figure  4  Ratings  of  Five  Hazardous  Thought  •  ,  i-m  eiR 
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Figure  5.  Ratings  of  stress  reduction  material. 
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Figure  6  Ratings  of  Poor  Judgment  Chain  concept 
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Figure  7.  Ratings  of  Six  Action  Ways  concept. 


Figure  8  Ratings  of  Three  Mental  Processes  of  Safe  Flight  concept 
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As  may  be  seen,  almost  half  (48%)  of  the  student  pilots  rate  the  Five 
Hazardous  Thoughts  concept  as  "very  useful".  The  concept  of  the  Poor 
Judgment  Chain  is  also  given  high  ratings  by  the  students,  with  42% 
considering  this  concept  to  be  "very  useful”.  However,  16%  of  the  student 
sample  indicate  that  these  two  concepts  have  been  of  little  or  no  value  in 
their  training.  Forty-four  percent  of  the  students  rate  the  chapter  on 
stress  reduction  as  "very  useful",  although  more  than  one  third  (36%)  place 
little  value  on  this  material.  The  Six  Action  Ways  and  the  Three  Mental 
Processes  of  Safe  Flight  are  given  markedly  lower  ratings  by  the  students. 
Although  one  fifth  (21%)  of  the  sample  view  the  Six  Action  Ways  as  "very 
useful",  an  equal  proportion  report  it  to  have  little  or  no  utility.  And  while 
one  third  (33%)  of  the  students  rate  the  Three  Mental  Processes  of  Safe 
Flight  as  “very  useful",  an  even  greater  proportion  (42%)  see  little  worth 
in  this  concept. 

Almost  two-thirds  of  the  instructor  sample  rate  both  the  stress 
reduction  chapter  (64%)  and  the  concept  of  the  Five  Hazardous  Thoughts 
(65%)  as  "very  useful".  No  low  ratings  at  all  are  given  to  the  stress 
reduction  material,  although  8%  of  the  instructors  consider  the  Hazardous 
Thoughts  to  be  only  "slightly  useful"  The  Poor  Judgment  Chain  model  is 
rated  as  "very  useful"  by  more  than  half  (55%)  of  the  instructors,  with  7% 
viewing  it  as  only  "slightly  useful".  Relatively  lower  ratings  are  qiven  to 
the  concepts  of  Three  Mental  Processes  of  Safe  Flight  and  the  Six  Action 
Ways  The  Mental  Processes  are  rated  "very  useful"  by  41%  of  the 
instructors,  with  12%  noting  that  they  are  "slightly  useful"  And  while 
over  one  third  (36%)  of  the  instructor  sample  consider  the  Action  Ways  to 
be  "very  useful",  an  almost  equal  proportion  (3 1  %)  see  only  slight  util ity  to 
this  concept 

The  ratings  assigned  to  each  component  by  each  respondent  were 
scaled,  and  the  mean  ratings  are  shown  in  Figure  9  Although  instructors 
gave  consistently  higher  ratings  to  each  component  than  did  the  student 
pilots,  there  was  genera!  agreement  as  to  the  relatively  greater  value  of 
the  stress  reduction  material,  the  Hazardous  Thoughts  concept,  and  the 
Poor  Judgment  Cham  model.  Respondents  also  agreed  on  the  lesser  value 
of  the  concepts  of  the  Three  Mental  Process  uf  Safe  Flight  and  the  ;.e 
Action  Ways,  and  their  utility  in  the  lodgment  training  preg:  am  u..hjU  be¬ 
quest  i  or  ied 
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Figure  9.  Mean  scaled  ratings  of  judgment  training  concepts 


Training  Time  Requirements 

The  study  was  also  concerned  with  the  amount  of  training  time  which 
the  judgment  program  consumed  within  the  private  pilot  curriculum  Not 
only  were  instructors  asked  to  master  the  judgment  training  material  on 
their  own  time,  but  they  also  had  to  allocate  ground  school  and  in-flight 
time  with  each  student  to  accomplish  the  program's  objectives 

Most  instructors  (44%)  reported  spending  a  total  of  between  two  and 
four  hours  of  self-study  time  preparing  to  teach  the  judgment  materials, 
while  almost  one  third  (29%)  spent  between  four  and  six  hours  in  this 
activity  Thus,  on  average,  the  program  appears  to  require  that  instructors 
devote  about  three  to  four  hours  of  self  study  time  in  order  to  become 
reasonably  familiar  with  the  contents 

Because  the  judgment  materials  were  designed  to  be  easily  mtedrated 
into  existing  flight  and  ground  instruction,  it  was  felt  fhat  reach  ‘  ‘he 
judgment  training  would  be  given  in  conjunct  -  or  with  item:  *ror-  tKe 
standard  pilot  training  curriculum  for  vtrm-  the  ’"st^rtcm  -  r.r 
caution  the  student  about  the  hazardous  thought  "-■u's.  h  At  :- 
describing  the  dangers  of  rushing  a  *  j.»  -  •  •  r\.r  tv:  - 
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instructors' estimates  of  ground  and  flight  times  devoted  to  judgment 
training  do  not,  per  se.  imply  any  additional  time  requirements. 

Instructors  also  reported  that  the  judgment  training  consumed  only 
one  or  two  hours  of  ground  time  (46%)  and  flight  time  (36%)  per  student. 
Only  a  small  proportion  of  instructors  reported  that  the  material  required 
more  than  four  hours  in  the  primary  flight  training  program.  These  results 
are  summarized  in  Figure  10. 

Most  students  (60%)  reported  that  they  spent  between  two  and  six 
hours  in  self-study  with  the  student  manual,  with  the  modal  time 
requirement  quite  close  to  four  hours.  Half  (50%)  reported  spending  less 
than  two  hours  of  ground  time  reviewing  judgment-related  material  with 
their  instructors,  and  58%  said  that  they  had  devoted  less  than  two  hours 
of  in-flight  time  to  such  matters  However,  it  is  not  at  all  clear  that  the 
student's  perception  of  what  was  and  what  was  not  "judgment-related" 
coincided  with  the  instructors' actual  objectives  in  any  given  instance 
These  findings  are  illustrated  in  Figure  1 1 
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Figure  10  Instructor  estimated  time  requirements  tor  ludqment  tram'’-! 
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Figure  1 1.  Student  estimated  time  requirements  for  judgment  training. 


Overall  Program  Evaluation 


Summary  guestions  helped  to  evaluate  the  degree  to  which 
participants  felt  that  the  program  had  accomplished  its  goals. 
Approximately  one  quarter  (24%)  of  the  instructors  said  that  "all"  of  their 
students  had  benefited  from  the  program,  and  almost  half  (46%)  indicated 
that  "most"  of  their  students  had  benefited  from  judgment  training.  More 
than  half  (56%)  of  the  students  reported  that  their  aeronautical  judgment 
had  "definitely"  improved  as  a  result  of  the  program,  while  one  fifth  (20%) 
were  uncertain  or  felt  that  their  judgment  had  not  improved.  About  one 
third  (32%)  of  the  instructors  felt  that  their  typical  student's  judgment 
had  "definitely"  improved  as  a  result  of  program  participation,  but  20% 
were  uncertain  or  saw  no  improvement 


An  important  question  asked  of  respondents  was  whether  they  would 
recommend  that  the  judgment  training  program  be  adopted.  Four-fifths  of 
the  student  sample  (80%)  recommended  program  adoption,  and  almost 
two-thirds  (63%)  "strongly  recommended"  the  use  of  the  judgment  manual 
in  private  pilot  training  Twenty-one  percent  of  the  students  were 
hesitant  or  would  not  recommend  the  program  These  results  are 
presented  in  Figure  12 
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STRONGLY  RECOMMEND 
RECOMMEND 
RECOMMEND  WITH  HESITATION 
DO  NOT  RECOMMEND 
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Figure  12.  Student  recommendations  regarding  judgment  training. 


More  than  three-quarters  (77%)  of  the  instructors  recommended  that 
the  program  be  adopted  on  a  voluntary  basis,  with  almost  half  (48%) 
"strongly"  recommending  such  adoption.  More  than  half  (55%)  of  the 
instructor  sample  recommended  that  judgment  training  be  required,  and 
about  one  quarter  (26%)  "strongly  recommended"  that  the  FAA  require  that 
such  a  program  be  used  to  teach  and  evaluate  judgment  skills  in  student 
pilots.  Still,  24%  of  the  instructors  were  hesitant  or  unwilling  to 
recommend  it  on  even  a  voluntary  basis,  and  almost  half  (45%)  of  the 
instructors  had  serious  reservations  about  mandatory  judgment  training. 
These  results  appear  in  Figure  13. 


Figure  1 3  Instructor  recommendations  regarding  judgment  training. 
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IMPLICATIONS  &  CONCLUSIONS 

Based  upon  the  findings  of  this  study,  the  following  conclusions  are 
offered; 

1)  The  Student  and  Instructor  Manuals  are  highly  acceptable  to  the 
user  community,  and  most  participants  found  them  to  be  very  useful. 

2)  Of  the  major  didactic  tools  used  in  the  program,  the  Five 
Hazardous  Thoughts  concept,  the  Poor  Judgment  Chain  model,  and  the 
stress  reduction  material  are  most  useful.  They  should  be  retained  in 
future  versions  of  the  judgment  training  program. 

3)  The  concepts  of  the  Six  Action  Ways  and  the  Three  Mental 
Processes  of  Safe  Flight  may  be  too  pedantic  for  general  use,  and  might  be 
dropped  from  subsequent  training  manuals. 

4)  Participating  students  and  instructors  felt  that  the  program 
improved  pilot  decision-making  skills. 

5)  Both  students  and  instructors  strongly  recommended  adoption  of 
the  program,  although  instructors  were  not  particularly  supportive  of 
required  judgment  training. 

6)  The  observation  flight  data  suggest  that  the  program  is  effective 
in  actually  improving  pilot  judgment,  even  in  the  informal  atmosphere 
which  characterizes  most  FBO  training  programs  However,  the  program's 
effectiveness  in  these  settings  may  be  considerably  less  dramatic  than 
was  suggested  by  the  earlier  studies 
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NOTES 

1.  Significant  contributions  to  this  study  were  made  by  Georgette  Buch 
(Transport  Canada),  Douglas  P.  Harvey  (FAA),  Russell  A.  Lawton  (AOPA),  and 
Gary  S  Livack  (GAMA).  The  observation  flights  were  conducted  by  AOPA 
staff  members  Robert  Gathers,  Thomas  Oneto,  Glenn  Rizner,  and  John 
Sheehan.  Research  assistance  at  Colby  College  was  provided  by  Karen  M. 
Barbera,  Deborah  E.  Burke,  Fran  M,  Gradstein,  and  Melissa  J.  Hruby. 
Secretarial  support  was  provided  by  Dorothy  Evertsen. 

2.  In  order  to  determine  the  extent  to  which  the  flight  training  activities 
presented  material  not  commonly  included  in  private  pilot  training,  an 
informal  small-sample  survey  was  conducted  at  three  flight  schools  which 
were  not  part  of  the  experiment  but  which  utilize  the  same  type  of 
traditional  pilot  training  curriculum  as  was  used  at  the  participating 
FBO's  The  results  suggest  that  only  12  of  the  the  50  Judgment  Training 
Scenarios  are  unique  to  the  program  and  present  activities  which  are 
rarely  undertaken  by  flight  instructors  in  normal  practice.  In  contrast,  21 
of  the  activates  presented  in  the  Scenarios  are  used  on  occasion  by  more 
than  half  of  the  flight  instructors  surveyed,  and  17  are  used  by  some 
instructors.  Of  the  activities  outlined  in  the  18  Lesson  Plans,  all  but  one 
are  commonly  included  in  flight  training  by  almost  all  instructors 
surveyed. 

3.  These  evaluations  were  subsequently  made  available  to  the  FAA 
researchers  involved  in  evaluating  the  prototype  sectional  charts 
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APPENDIX  A 

Participating  Fi*ed  Base  Operators 


City 

State 

Airoort 

Fixed  Base  Operator 

ID 

Middletown 

DE 

Summit 

Summit  Aviation 

N92 

Frederick 

MD 

Frederick 

Frederick  Aviation 

FDK 

Farminqdale 

NY 

Republic 

Fliqhtways  of  Long  Island 

FRO 

1  si  ip 

NY 

IsliD-Mac  Arthur 

Mid-Island  Air  Service 

ISP 

Allentown 

PA 

Queen  City 

Queen  City  Aviation 

1N9 

Easton 

PA 

Easton 

Bradens  Flying  Service 

N43 

Chesapeake 

v'A 

Chesapeake 

Mid-Eastern  Airways 

W36 

Lynchburg 

VA 

Lynchburg 

Air  Virginia 

LYH 

r  lanassas 

VA 

Manassas 

Dulles  Aviation 

W10 

Woodbridqe  Woodbridqe  Aviation  W22 


Woodbridqe  VA 
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APPENDIX  B 

Observation  Flight 
Administration  &  Scoring  Guidelines 

L  Flight  Information  During  the  mission  preview,  the  observer  indicates 
to  the  pilot  that  the  flight  will  proceed  to  a  specific  destination  airport, 
if  possible,  this  field  should  be  one  which  has  a  currently  valid  NOTAM 
and/or  one  with  which  the  pilot  is  relatively  unfamiliar.  As  the  observer 
and  pilot  later  walk  to  the  aircraft,  the  observer  will  casually  inquire  if 
there  are  any  NOT AMs  on  the  destination  airport  and/or  if  the  weather  will 
be  all  right  for  the  flight. 

Good  Judgment:  pilot  indicates  that  he  has  checked  an  appropriate  source 
for  NOT AMs  and/or  weather 

Poor  Judgment:  pilot  fails  to  check  NOTAMs  and/or  weather  prior  to 
boarding  the  aircraft 


2.  Aircraft  Condition  Prior  to  the  time  that  the  pilot  arrives  at  the 
airport,  or  while  he  is  completing  the  aeronautical  chart  survey  form,  the 
observer 

a)  loosens,  but  does  not  remove,  one  of  the  fuel  tank  caps 

b)  loosens,  but  does  not  remove,  the  oil  fill  cap/dip  stick 

c)  pulls  a  fuse  or  circuit  breaker 

Good  Judgment  the  pilot  detects  and  remedies  the  defect  during  the 
preflight  inspection  prior  to  boarding  aircraft 

Poor  Judgment,  the  pilot  fails  to  detect  and  remedy  the  defect  prior  to 
boarding  aircraft 

NOTE.  IF  UNDETECTED  BY  THE  PILOT,  THE  OBSERVER  MUST  REMEDY  THE 
DEFECT  PRIOR  TO  BOARDING  AIRCRAFT 


3  Cabin  Hazard  Prior  to  the  time  that  the  pilot  arrives  at  the  airport,  or 
while  he  is  completing  the  aeronautical  chart  survey  form,  the  observer 
places  a  completely  empty  soft  drink  bottle  on  the  cockpit  floor  in  front  of 
the  right  seat  halfway  between  the  front  of  the  seat  and  the  rudder  pedals 
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Good  Judgment:  pilot  detects  and  safely  secures  the  bottle  during  the 
preflight  inspection  prior  to  engine  start 

Poor  Judgment;  pilot  fails  to  detect  and  secure  the  bottle  prior  to  engine 
start 

NOTE:  IF  UNDETECTED  BY  THE  PILOT,  THE  OBSERVER  MUST  SAFELY  SECURE 
THE  BOTTLE  WITHOUT  COMMENT  PRIOR  TO  TAXI 


4.  Propeller  Hazard  Approximately  30  seconds  after  engine  start,  but 
before  taxi  or  run-up,  the  observer  releases  his  seat  belt  as  he  announces 
that: 

a)  he  has  dropped  his  pen  in  front  of  the  aircraft 

b)  he  sees  a  loose  hold-down  on  the  cowling 

c)  he  sees  a  nail  on  the  runway  in  front  of  the  aircraft 
He  then  begins  to  open  the  door  to  exit  the  aircraft 

Good  Judgment:  pilot  immediately  shuts  down  the  engine  or  requests  that 
the  observer  remain  in  the  aircraft 

Poor  Judgment:  pilot  permits  the  observer  to  leave  the  aircraft  while 
engine  is  running 

NOTE  OBSERVER  MUST  NOT  EXIT  THE  AIRCRAFT  WITH  ENGINE  RUNNING 


5.  Checklist  Distractions  During  the  execution  of  the  prestart,  pretaxi, 
and/or  run-up  checklist  the  observer  converses  with,  questions  or 
otherwise  distracts  the  pilot  in  such  a  wav  as  to  interrupt  the  systematic 
execution  of  the  checklist.  Two  interruptions  are  to  be  made 

Good  Judgment,  pilot  requests  that  the  observer  hold  his  questions  and 
comments  until  after  the  checklist  is  completed,  or  pilot  ignores 
observer's  remarks 


Poor  Judgment  pilot  interrupts  the  checklist  to  converse  with  the 
observer,  or  carries  on  the  conversation  while  continuing  the  checklist 
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6.  Departure  Procedures  The  observer  requests  one  of  the  following 
non-standard  departure  procedures 

a)  a  takeoff  from  a  marginally  safe  intersection 

b)  a  right  (or  non-standard  left)  turn  immediately  after  takeoff  while 
still  in  the  pattern  at  an  uncontrolled  field 

c)  a  low  (200  Feet  AGL)  turn  immediately  after  takeoff 

Good  Judgment,  pilot  declines  request 

Poor  Judgment:  pilot  executes  potentially  hazardous  departure  procedure 

NOTE.  OBSERVER  MUST  NOT  PERMIT  THE  PILOT  TO  EXECUTE  A  HAZARDOUS 
DEPARTURE 


7.  Low  Flight  Over  Obstruction  The  observer  requests  a  flight  path  and 
altitude  which  will  take  the  aircraft  barely  1,000  feet  above  some  obiect 
or  building  in  a  built-up  area 

Good  Judgment:  pilot  notes  conflict  and  either  climbs  or  diverts  in  order 
to  pass  the  object  at  a  safe  distance 

Poor  Judgment:  pilot  flies  low  over  vertical  obstruction 


NOTE  OBSERVER  MUST  NOT  PERMIT  HAZARDOUSLY  LOW  FLIGHT  OVER 
OBSTRUCTION 


8.  Low  Steep  Turn  Over  Building  The  observer  requests  that  the  pilot 
circle  above  a  certain  building  so  that  the  observer  can  take  a  photograph 
The  observer  requests  an  increasingly  steeper  turn  until  a  45  deqree  angle 
of  bank  is  established 

Good  Judgment  pilot  climbs  to  a  safe  altitude,  or  does  not  exceed  a  30 
deqree  angle  of  bank  at  low  attitude,  or  declines  request 

Poor  Judgment  pilot  attempts  to  execute  a  hazardous  low  steep  turn 

NOTE  OBSERVER  MUST  NOT  PERMIT  A  HAZARDOUSLY  LOW/STEEP  TURN 
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9.  Low  Flight  Over  Unsuitable  Terrain  The  observer  requests  a  flight 
path  and  altitude  over  unsuitable  terrain  such  that  the  ability  to  execute  a 
safe  emergency  landing  or  power-off  glide  to  shore  would  be  senous'y 
compromised. 

a)  heavily  wooded  terrain 

b)  mountainous  terrain 

c)  large  body  of  water 

Good  Judgment  pilot  declines  request  or  climbs  to  a  safe  altitude 

Poor  Judgment  pilot  flies  low  over  unsuitable  terrain 

NOTE  OBSERVER  MUST  NOT  PERMIT  HAZARDOUSLY  LOW  -LIGHT  OVER 
UNSUITABLE  TERRAIN 


IQ  VFR  Cruising  Altitude  The  observer  requests  a  fiiqht  pats  and 
altitude  consistent  with  VFR  cruising  altitude  regulations,  and  requests 
the  pilot  to  locate  a  specific  terrain  feature  along  this  flight  path  upon 
reaching  the  terrain  feature,  the  observer  requests  the  pilot  to  f’y  a  new 
heading  such  that  a  climb/descent  is  required  in  order  to  maintain  an 
abbroonate  VFR  cruising  altitude  The  observer  mates  no  mention  o' 
altitude  however 

Good  Judgment  pilot  notes  conflict  anq  eitM  <_ i 'rnp-  m  ge^eM:  to  at 
appropriate  VFR  cruising  altitude 

Poor  .Judgment  pilot  turns  to  the  new  heading  but  t a. :  .  vy  a ' t  ■ 

to  comply  with  vpr  cruising  altitude  reQulanons 

NOTE  OBSERVER  MlJoT  NOT  PERMIT  A  ha/aR .  a  ,  ■  A ' . \  :  ,■ : *’■ 

ORuiSlNG  ALTiTIJDE  REGULATIONS 


1  i  Controlled  Airspace  E ncroachmont 
path  and  air  dude  wmcn  brings  the  a 
horizontal  or  vertical  Om as  of  v,  ck '  v- 
control  Arr,a.  whi  ie  M-e  jir  * 

r  ontr  .jl  1  :nq  ' aC  hot 
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Good  Judgment  pilot  informs  observer  that  the  requested  flight  path 
cannot  be  flown  safely  without  ATC  clearance  or  pilot  requests  such 
clearance  from  ATC,  or  pilot  alters  flight  path  to  avoid  the  controlled 
airspace  by  a  safe  margin 

Poor  Judgment  pilot  flies  marginally  close  to  an  Active  Airport  Traffic 
Area  or  Terminal  Control  Area  without  ATC  Clearance 

NOTE  OBSERVER  MUST  NOT  PERMIT  THE  AIRCRAFT  TO  ENTER  INTO 
CONTROLLED  AIRSPACE  WITHOUT  ATC  CLEARANCE 


12 _ Arrival  Procedures  On  approach  to  an  uncontrolled  airport,  the 

observer,  in  or ciei  to  take  a  photograph,  requests  a  flight  path  and  altitude 
which  brings  the  aircraft. 

a)  directly  over  the  airport  at  pattern  altitude  but  not  in  the  pattern 
to  at  pattern  altitude  but  flying  a  non-standard  pattern 
rj  set  up  high  on  the  final  approach  leq 
di1  set  up  for  a  downwind  landing 

Good  Judgment  pilot  informs  oPserver  of  the  need  to  fly  a  standard 
approach  and  landing  pattern,  and  flies  it 

Puor  Judgment  pilot  flies  a  non-standard  approach  or  landing  pattern  at 
pattern  altitude 

NOTE  OBSERVER  MUST  NOT  PERMIT  A  HAZARDOUS  APPROACH  OR  LANDING 


•w- 
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APPENDIX  C 
Student  Evaluation 

The  Federal  Aviation  Administration  (FAA)  and  the  General  Aviation 
Manufacturers  Association  asked  several  flight  schools  to  examine  a 
prototype  Pilot  Judgment  Training  Program  We  understand  that  you 
participated  in  this  important  program,  and  we  would  be  interested  in  your 
views  on  its  acceptability  Please  circle  your  response  and  feel  free  to 
explain  it  below  the  question,  or  on  the  back  of  the  page  if  necessary  A 
self-addressed,  postage  paid  envelope  is  enclosed  for  your  convenience 
Your  cooperation  in  this  critique  is  deeply  appreciated 


1  The  Judgment  Training  Manual  in  general  was 

A)  very  useful  B)  moderately  useful 

C.)  slightly  useful  D)  not  at  all  useful 

2  Six  Action  Ways  concept  (Do,  No  Do,  Under  Do,  Over  Do,  Early  Do,  Late 
Do)  was 

A)  very  useful  B)  moderately  useful 

C)  slightly  useful  D)  not  at  all  useful 

3  Poor  Judgment  Cham  concept  was 

A)  very  useful  B)  moderately  useful 

C)  slightly  useful  D)  not  at  all  useful 

4  Three  Mental  Processes  of  Safe  Fhqht  concept  (Automatic  Reaction, 
Problem  Resolving,  Repeated  Reviewing)  were 

A)  very  useful  B)  moderately  useful 

C)  si ightly  useful  D)  not  at  all  useful 

5  Five  Hazdidous  Thoughts  (Anti-authority,  Impulsivity,  invulnerability, 
Macho.  Resignation)  were 

A)  very  useful  B)  moderately  useful 

C)  slight  ly  useful  D)  not  at  all  useful 

G  Chapter  on  Identifying  and  Reducing  Stress  was 

A)  very  useful  B)  moderately  useful 

C)  slightly  useful  D)  not  at  all  useful 
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7  How  much  self-study  time  did  you  spend  on  the  Judgment  Training 
Manual  ? 

A)  less  than  2  hrs  B)2-4hrs.  C)4-6hrs 

D)  6-8  hrs  E)  more  than  8  hrs 

8.  How  much  total  time  did  your  instructor  spend  discussing  judgment 
with  you  in-flight7 

A)  less  than  2  hrs  B)  2-4  hrs  0  4-6  hrs 

D)  6-8  hrs  E)  more  than  8  hrs 

9.  How  much  total  time  did  your  instructor  spend  discussing  judgment 
with  you  on  the  ground7 

A)  less  than  2  hrs  B)  2-4  hrs.  C)  4-6  hrs. 

D)  6-8  hrs  E)  more  than  8  hrs 

10  Do  you  feel  that  reading  this  manual  and/or  discussing  these  concepts 
improved  your  aeronautical  judgment7 

A)  definitely  B)  probably 

C)  uncertain  D)  no 

1 1  Would  you  recommend  the  adoption  of  the  Judgment  Manual  for  use  in 
private  pilot  training7 

A)  strongly  recommend  B)  recommend 

C)  recommend  with  hesitation  D)  do  not  recommend 

12  How  could  this  manual  be  improved7 
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APPENDIX  D 
Instructor  Evaluation 

1  Instructor  Manual  in  general  was 

A)  very  useful  B)  moderately  useful 

C)  slightly  useful  D)  not  at  all  useful 

2  In-Flight  "Lesson  Plans"  and  "Judgment  Training  Scenario:  •'  *'■*- 
instructor  manual  were 

A)  very  useful  B)  moderately  useful 

C)  slightly  useful  D;  not  at  all  useful 

3  Student  Manual  in  general  was 

A)  very  useful  B)  moderately  useful 

C )  slightly  useful  D)  not  at  all  useful 

4  Six  Action  yyays  concept  (Do.  No  Do.  under  Do.  Over  Do,  Early  jo  .  o- 
Do)  was 

A)  very  useful  B)  moderately  useful 

O  slightly  useful  jj  nut  at  all  useful 

Pour  Judament  Chain  concepf  was 

A)  very  useful  L:.-  moderated'  use’  j 

C )  sughtiv  useful  [;,■  not  at  all  usefu: 

o  Three  Mental  Processes.  of  Safe  Flight  . onrept  Ai/cinvit e.  ~~ *  * 

P'  oh l err i  Reso'v'nq,  Repeated  Reviewing'1  we>  e 

A;  very  useful  6)  mode'atew  .ot-',, 

L;  Slightly  useful  0  not  at  ail  j:,et.ji 

Five  HararoouS  Thoughts  vAnt 
Mac  no.  Res ■  gnat  ion;  were 
A)  very  useful 
C  )  slight  iy  uset ijl 

8  ,*i  ess  r euogfi  t  ion  chapter  wa  • 

A  » »  u  . t-* r  ,  i 
r  ;  -I  ■  gnt  ■ ,  u -,et 
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9  How  much  self-study  time  did  you  spend  (reading  the  Student  and 
instructor  Manuals,  planning  in-flight  exercises,  etc.)  in  order  to  permit 
you  to  teach  the  judgment  concepts'?  Do  NOT  include  the  time  spent  in  the 
training  sessions  with  FAA  scientists 

A)  less  than  2  hrs  B)2-4hrs  C)  4-6  hrs 

D)  6-3  hrs  E)  more  than  8  hrs 


iQ  Apout  how  much  ground  time  did  you  spend  teaching  judgment  to  each 
of  /our  students'? 


A )  None 
O  :  2-4  hrs 


B)  less  than  1  hr 
E)  more  than  4  hrs. 


C)  1-2  hrs 


:  i  Apout  how  much  flight  time  did  you  spend  teaching  judgment  to  each 
of  v'  'j  :>t udei its  1 

a  None  B)  less  than  1  hr  C)  1-2  hrs 

;  _  -4  nrs  E)  more  than  4  hrs 


a  mid  you  recommend  the  voluntary  use  of  these  manuals  for  private 

;  •  •  dm  ng? 

A >  strongly  recommend  B)  recorrirnend 

2  i  recommend  with  hesitation  D)  do  not  recommend 


,  rj  recommend  mat  FAA  require  a  program  such  as  this  to  teacn 
jiojte  judgment  sf  ills  of  student  pilots’'1 

a.  strongly  recommend  B)  recommend 

.' )  recommend  with  hesitation  D)  do  not  recommend 


you  thoi-  part u: '.pat. mg  m  this  program  improved  vdur  typical' 
r  'jggmer,:  sm  1 1:. ’• 


A  /  ilef  i  n ;  f  e  i  / 


B) probapi y 


••  i- 1  •  .i'  e 


J)  I'iO 


f  f  i  n  f  f  : ,  •  r 


■  ir  :  *  A*' ('  ;  :L  J '  m ,1  i!i  r  m LV 


=  n  •  * '  ■  t » 
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Control  Group  Data 


Observation 

No.  Observer  Flight  Score  Age 


.1  i.|  «| 


Judgment  Training 


Page  36 


Observer 
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Experimental  Group  Data 


Observation 
Flight  Score 


